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Rising employer demand for skilled workers has driven 
efforts to better align occupational training programs to 
industry needs.! Yet, even as the demand for skilled workers 
increases, less than half of students who enter occupational 
training programs receive a credential within six years.” 
Community colleges are working to find faster and more 
effective ways to train those in need of basic skills instruc- 
tion in math, reading, or job skills. Traditionally, basic 

skills courses are offered in a sequence that must be com- 
pleted before students can begin college-level occupational 
training. However, most students referred to basic skills 
training never enroll in college-level courses.’ As its name 
implies, Washington State’s Integrated Basic Education Skills 
and Training (I-BEST) provides integrated basic skills and 
occupational training that allows students to complete their 
training program faster, and provides supports designed to 
ensure students stay engaged in training. Washington State’s 
I-BEST program was developed by the Washington State 
Board of Community and Technical Colleges (SBCTC) and 
was first implemented in the 2006-2007 school year.* Since 
its creation, I-BEST has been replicated in other locations, 
sometimes under different names. Accelerating Opportunity 


was launched in four states in 2011 with funding from the 
Bill & Melinda Gates Foundation. With support from the U.S. 
Department of Labor’s Workforce Innovation Fund (WIF), 
the Accelerating Connections to Employment (ACE) program 
was implemented in four states in 2013. Both Accelerating 
Opportunity and ACE programs note that they are based 

on the I-BEST model, with the same core commitment to 
integrated basic skills and occupational training. Supports 
provided by these programs differ slightly, and these differ- 
ences are described in this report. 


This What Works Clearinghouse™ (WWC) report, part of 
the WWC’s Postsecondary Career and Technical Education 
topic area, explores the effects of I-BEST on education and 
labor market outcomes. The WWC identified 12 studies of 
I-BEST. Three of these studies meet WWC standards.° The 
evidence presented in this report is from studies of the 
impact of I-BEST on students in career and technical educa- 
tion programs—including African-American, Hispanic, Asian, 
Native Hawaiian, and White students—in a variety of school 
settings, including urban, suburban, and rural community 
colleges. 


What Happens When Students Participate in I-BEST?® 


The evidence indicates that implementing I-BEST: 

¢ is likely to increase industry-recognized credential, 
certificate, or license completion 

¢ may increase short-term employment 

¢ may increase short-term earnings 


e may result in little or no change in credit 
accumulation 


Findings on I-BEST from three studies that meet WWC 
standards are shown in Table 1. The table reports an effec- 
tiveness rating, the improvement index, and the number of 
studies and students that contributed to the findings. The 
improvement index is a measure of the intervention’s effect 
on an outcome. It can be interpreted as the expected change 
in percentile rank for an average comparison group student 
if that student had received the intervention. 


Table 1. Summary of findings on I-BEST from studies that meet WWC Standards 


Study findings Evidence meeting WWC standards (version 4.0) 
Improvement index 
Outcome domain Effectiveness rating (percentile points) Number of studies Number of students 
“oTHfoea a7 Paice Sp Positive effects H18 3 44,367 
Short-term employment Potentially positive effects +10 1 2,064 
Short-term earnings Potentially positive effects 0 2 2,519 
Credit accumulation No discernible effects -1 1 42,894 


Note:The improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the 
intervention. For example, an improvement index of +18 means that the expected percentile rank of the average comparison group student would increase by 18 points if the 
student received /-BEST. The improvement index values are generated by averaging findings from the outcome analyses that meet WWC standards, as reported by Glosser 
et al. (2018), Modicamore et al. (2017), and Anderson et al. (2017). A positive improvement index does not necessarily mean the estimated effect is statistically significant. 
Industry-recognized credential, certificate, or license completion outcomes reported in these studies include receipt of a credential from any source; receipt of a vocational, 
technical, or professional license or certificate; and receipt of any credential from a college. The short-term employment outcome reported in one study was employed in 

the first year after program completion. Short-term earnings outcomes included working in a job paying $12 or more after 18 months and earnings one year after program 
completion. The credit accumulation outcome was the percentage of students earning more than 12 credits. The effects of -BEST are not known for other outcomes within 
the Postsecondary Career and Technical Education topic area, including technical skill proficiency, postsecondary degree attainment, medium-term employment, long-term 
employment, medium-term earnings, and long-term earnings. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the Postsecondary Career and Technical Education topic area protocol 
(version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship 
between effectiveness ratings and the statements used in this report: 


Effectiveness rating Rating interpretation Description of the evidence 


Positive (or negative) effects The intervention is likely to change an Strong evidence of a positive effect, with no 
outcome overriding contrary evidence 


Potentially positive (or negative) effects The intervention may change an outcome Evidence of a positive effect with no overriding 
contrary evidence 


No discernible effects The intervention may result in little to no No affirmative evidence of effects 
change in an outcome 


Mixed effects The intervention has inconsistent effects Evidence includes studies in at least two of 
onan outcome these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


How is I-BEST Implemented? 


The following section provides details of how I-BEST was 
implemented. This information can help educators identify 
the requirements for implementing I-BEST and determine 
whether implementing this intervention would be feasible 
in their colleges. Information on I-BEST presented in this 
section comes from the studies that meet WWC standards 
(Glosser et al., 2018; Modicamore et al., 2017; and Anderson 
et al., 2017) and from correspondence with the developer. 


Comparison group: In the three studies that 
contribute to this intervention report, students in 
the comparison group received the regular supports 
provided by their community college. In the ACE 


study (Modicamore et al., 2017), students who entered 
the program through a WIB had access to regular WIB 
services which included training referrals, career 
counseling, and job search assistance. 


N 


Goal: I-BEST was developed by SBCTC to increase the 
rate at which adults in need of basic skills enter and 
succeed in postsecondary occupational training. It is 
designed to integrate adult basic education and occu- 
pational skills training, so students can learn literacy, 
math, work, and college-readiness skills and move into 
living wage jobs faster. It provides an alternative to the 
traditional track of providing adult basic education 
prior to students entering occupational training, which 
generally results in low rates of advancement between 
basic skills and occupational training. 


Target population: I-BEST allows individuals with skill 
levels that are lower than normally required to enroll in 
college-level programs to pursue credit-bearing, short- 
term certificate programs as well as college degrees. 


Method of delivery: I-BEST’s signature feature is its 
team teaching approach, which involves a basic skills 
instructor and an occupational instructor co-teaching 
during at least 50 percent of occupational training class 
time. The Accelerating Opportunity model calls for a 
minimum of 25 percent team teaching, while the ACE 


model prescribes a 50 percent minimum of team teach- 
ing. In addition, a dedicated I-BEST navigator (coach) is 
available to students who can provide career counseling 
and help students access “fill-the-gap” financial support 
for tuition and course materials; funding for support 
services (e.g., uniforms, transportation, licensure 
testing); clinical placements (for nursing students); and 
internships. Both Accelerating Opportunity and ACE had 
partnerships with Workforce Investment Boards (WIBs) 
that connected students to employers. 


Frequency and duration of service: Some I-BEST 
programs require full-time enrollment while others 

are offered part-time in the evenings or weekends. The 
duration of students’ participation in I-BEST depends 

on their course of study. For example, as reported in 
Glosser et al. (2018), automotive, electrical, and certified 
nursing assistant trainings lasted one quarter while pre- 
cision machining, welding, and sustainable office skills 
trainings lasted two quarters. 


Intervention components: Refer to Table 2 for addi- 
tional details. 


Table 2. Components of I-BEST 


Key component Description 


Team teaching _/-BEST offers students integrated basic skills and occupational skills training. Both the basic skills instructor and the occupational 
training instructor are required to be present in class for at least half of the total instructional time in an /-BEST course—or 25 percent 
of total instructional time in Accelerating Opportunity. Instructors collaborate to identify joint learning outcomes for students in their 
class and both take part in leading discussions and managing student projects. Team teaching can take on different forms: 
¢ Traditional team teaching, where two or more teachers share instructional responsibilities in the same classroom at the same 
time with the same group of students. 

* Collaborative teaching, where teachers exchange and discuss ideas in front of learners, instead of engaging in usual direct 
instruction. 

¢ Complementary-supportive teaching, where one teacher is responsible for teaching content and the other is responsible for 
providing follow-up activities or study skills. 

¢ Parallel instruction, where the class is divided into two groups and each teacher is responsible for teaching the same material to 
each group. 

¢ Differentiated split class, where the class is divided into smaller groups according to learning needs, and each teacher provides 
instruction to their respective group. 

¢ Monitoring teacher, where one teacher instructs the entire class and the other teacher circulates in the classroom and monitors 
student understanding and behavior. 


In the Accelerating Opportunity evaluation (Anderson et al., 2017), all six types of team teaching models were used, with the 
complementary-supportive model being most popular, followed by monitoring teacher, traditional, and collaborative. In the ACE study 
(Modicamore et al., 2017), the complementary-supportive model was described as being typical. In the -BEST study (Glosser et al., 
2018), collaborative teaching was implemented, as were two variants of traditional team teaching: (1) basic skills instructors delivered 
instruction for part of the class period, then turned over instruction to the occupational skills instructor, and (2) both instructors 
delivered class content together. 


I-BEST uses a contextualized instruction approach, where students learn basic skills in the context of their course of study. For 
example, in an /-BEST nursing program, increased emphasis is placed on learning medical terms in addition to mastering everyday 
vocabulary. This instructional model aims to improve the motivation and achievement of students by providing them experiences 
where they can see the usefulness of basic skills instruction in their chosen field. 


Career I-BEST provides students with a career navigator, who provides intake, orientation, job readiness, mentoring, and job placement 

navigation services. /-BEST offers multiple tracks from its traditional program, to either provide additional vocational education that can lead to a 
college degree, or provide additional academic instruction to help students advance on a career pathway. 

Financial I-BEST students in Washington can receive Opportunity Grants when they enroll in -BEST professional technical pathways. The 

supports grant covers up to 45 credits of tuition and up to $1,000 a year for books and supplies. 

Additional I-BEST students also receive tutoring, career advising, emergency child care, emergency transportation, and college success 

supports classes. 


Job placement —_Both the Accelerating Opportunity and ACE implementation of the /-BEST model included partnerships with local WIBs to place 
assistance students in jobs. /-BEST as studied in Glosser et al. (2018) did not include specific employment and job placement services. 


What Does I-BEST Cost? 


This preliminary list of costs is not designed to be 
exhaustive; rather, it provides educators with an overview 
of the major resources needed to implement I-BEST. The 
program costs described below are based on information 
available as of March 2020. The total cost of I-BEST was 
reported in a SBCTC cost-benefit analysis to be $2,417 in 
direct student costs and $7,279 in state costs as of January 
2013. The total cost of ACE as of May 2017 ranged from 
$4,828 to $13,033 per student across the nine sites. The total 
cost of Accelerating Opportunity as of November 2017 ranged 
from $2,635 to $7,128 per student across four states. Below is 
a breakdown of the costs reported in the SBCTC study. 


¢ Equipment and materials costs: The SBCTC cost- 
benefit analysis reported that enrollment support, which 
includes both direct and indirect costs of instruction, was 
$4,396 per I-BEST completer. Washington State provided 
an average of $2,883 in financial aid support per I-BEST 
completer. 

¢ Personnel costs: The SBCTC cost-benefit analysis 
reported that enrollment support, which includes both 
direct and indirect costs of instruction, was $4,396 per 
I-BEST completer. 


For More Information: 


About /-BEST 
Washington SBCTC 
1300 Quince St SE, 4th floor 
Olympia, WA 98504-2495 


Attn: William Durden, Policy Associate, Basic Education for Adults 


¢ Facilities costs: No additional facilities costs were 
reported beyond the facilities costs normally associated 
with college attendance. 


¢ Costs paid by students or parents: The SBCTC cost- 
benefit analysis reported that tuition minus the weighted 
average financial aid per student completer was $1,114. 
The cost of books was listed as $1,000 and the cost of 
fees was $300 per student completer. Students can 
use Opportunity Grants when they enroll in E-BEST 
professional technical pathways, which cover up to 45 
credits of tuition and up to $1,000 a year for books and 
supplies. 


¢ In-kind supports: [-BEST students also receive tutoring, 
career advising, emergency child care, emergency 
transportation, and college success classes. 


¢ Sources of funding: I-BEST is funded by SBCTC. The 
Accelerating Opportunity program was funded by the Bill & 
Melinda Gates Foundation. The ACE program was funded 
through a U.S. Department of Labor Workforce Innovation 
Fund grant with additional support from the Annie E. 
Casey Foundation. 


Email: wdurden@sbctc.edu Web: https://www.sbctc.edu/colleges-staff/programs-services/i-best/ Phone: (360) 704-4368 


About the cost of the intervention 


Web: httos://www.sbctc.edu/resources/documents/colleges-staff/programs-services/basic-education-for-adults/ 


Investmentsin|-BESTPrograms. pdf 


See also 


Modicamore, D., Lamb, Y., Taylor, J., Takyi-Laryea, A., Karageorge, K., & Ferroggiaro, E. (2017). Accelerating Connections to 
Employment, volume 1: Final evaluation report. Fairfax, VA: ICF International. 


Kuehn, D., Anderson, T., Lerman, R., Eyster, L., Barnow, B., & Briggs, A. (2017). A cost-benefit analysis of Accelerating Opportunity. 
Washington, DC: Urban Institute. Retrieved from https:/Avww.urban.org/sites/default/files/publication/94876/ao-cba-report.pdf 


Research Summary 


The WWC identified 12 studies that investigated the effective- 
ness of I-BEST (Figure 1): 


¢ Two studies meet WWC group design standards without 
reservations 


¢ One study meets WWC group design standards with 
reservations 


¢ Three studies do not meet WWC group design standards 


¢ Six studies are ineligible for review 


The WWC reviews findings on the intervention’s effects on 
eligible outcome domains from studies that meet standards, 
either with or without reservations. Based on this review, 


the WWC generates an effectiveness rating, which summa- 
rizes how the intervention impacts, or changes, a particular 
outcome domain. The WWC reports additional supplemen- 
tal findings, such as state-by-state results for credit accumu- 
lation (Anderson et al., 2017), on the WWC website 

(https: //whatworks.ed.gov). 


These supplemental findings and findings from studies that 
either do not meet WWC standards or are ineligible for 
review do not contribute to the effectiveness ratings. 


The three studies of I-BEST that meet WWC group design 
standards reported findings on industry-recognized cre- 
dential, certificate, or license completion; short-term 


employment; short-term earnings; and credit accumulation. 
No other findings in the studies meet WWC group design 
standards within any outcome domain included in the 
Postsecondary Career and Technical Education topic area. 
Citations for the three studies reviewed for this report are 


Figure 1. Effectiveness ratings for I-BEST 
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I-BEST has positive effects on industry-recognized credential, certificate, or license completion 


The WWC determined that two studies that meet WWC group design standards without reservations 

(Glosser et al., 2018; Modicamore et al., 2017) and one study that meets WWC group designs standards with 
reservations (Anderson et al., 2017) showed evidence of a positive and statistically significant effect of -BEST on 
industry-recognized credential, certificate, or license completion. 


I-BEST has potentially positive effects on short-term employment 


The WWC determined that one study that meets WWC group design standards without reservations (Modicamore 
et al., 2017) showed evidence of a positive and statistically significant effect of -BEST on short-term employment. 


I-BEST has potentially positive effects on short-term earnings 


The WWC determined that one study that meets WWC group design standards without reservations (Modicamore et 
al., 2017) showed evidence of a positive and statistically significant effect of /-BEST on short-term earnings, and one 
study that meets WWC group design standards without reservations (Glosser et al., 2018) showed evidence of an 


indeterminate effect of /-BEST on short-term earnings. 


I-BEST has no discernible effects on credit accumulation 


The WWC determined that one study that meets WWC group designs standards with reservations showed evidence of 
an indeterminate effect of |-BEST on credit accumulation (Anderson et al., 2017). 


Main Findings 


Table 3 shows the findings from the three I-BEST studies 
that meet WWC standards. The table includes WWC calcu- 
lations of the mean difference, effect size, and performance 
of the intervention group relative to the comparison group. 
Based on findings from the three studies that meet WWC 
standards, the effectiveness rating for the industry-recog- 
nized credential, certificate, or license completion domain is 
positive effects, indicating strong evidence of a positive effect 
with no overriding contrary evidence. These findings are 
based on 44,367 students. The effectiveness rating for 


the short-term employment domain is potentially positive 
effects, indicating evidence of a positive effect with no 
overriding contrary evidence. This finding is based on 2,064 
students. The effectiveness rating for short-term earnings is 
potentially positive effects, indicating evidence of a positive 
effect with no overriding contrary evidence. These findings 
are based on 2,519 students. The effectiveness rating for the 
credit accumulation domain is no discernible effects, 
indicating no affirmative evidence of effects. This finding is 
based on 42,894 students. 


Table 3. Findings by outcome domain from studies of I-BEST that meet WWC Standards 


Mean 
(standard deviation) WWC calculations 
Sample Intervention Comparison Mean Effect § Improvement 
Measure (study) Study sample size group group difference _ size index p-value 
Received a credential from any Full Sample 424 33.5 18.2 15.3 0.49 +19 <.01 
source (%) (Glosser et al., 2018)? 
Earned a vocational, technical, or Full Sample 1,049 53.5 35.4 18.1 0.45 +17 <.01 
professional certificate or license 
(%) (Modicamore et al., 2017)° 
Received any credential from Full Sample 42,894 52.6 33.0 19.6 0.49 +19 <.01 
a college (%) (Anderson et al, 
2017)° 
Outcome average for industry-recognized credential, certificate, or license completion across all studies 0.48 +18 
Employed in the first year Full Sample 2,064 62.6 51.9 10.7 0.27 +10 < 01 
after program completion (%) 
(Modicamore et al., 2017)° 
Outcome average for short-term employment across all studies 0.27 +10 
Working in a job paying $12 an Full Sample 455 23.0 23.8 -0.8 -0.03 -1 85 
hour or more (%) (Glosser et al., 
2018) 
Earnings one year after program = Maryland and 1,513 $12,897.00 $11,601.80 $1,295.20 0.13 +5 <.01 
(Modicamore et al., 2017)? Texas Sample 
Earnings one year after program Connecticut 348 $14,125.19 $12,578.79 $1,546.40 0.15 +6 12 
(Modicamore et al., 2017)° Sample 
Earnings one year after program Georgia 203 $5,783.50 $7,154.60 -$1,371.10  -0.22 9 08 
(Modicamore et al., 2017)? Sample 
Outcome average for short-term earnings across all studies 0 0 
Earned more than 12 credits (%) Full Sample 42,894 45.7 46.9 -1.2 -0.03 -1 13 
(Anderson et al., 2017)° 
Outcome average for credit accumulation across all studies -0.03 -1 


Notes: For mean difference and effect size values reported in the table, a positive number favors the intervention group and a negative number favors the comparison group. 
The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are given the 
intervention (measured in standard deviations of the outcome measure). An indicator of the effect of the intervention, the improvement index can be interpreted as the expected 
change in percentile rank for an average comparison group student if that student had received the intervention. For example, an improvement index of +4 means that the 
expected percentile rank of the average comparison group student would increase by 4 points if the student received /-BEST. A positive improvement index does not 
necessarily mean the estimated effect is statistically significant. Some statistics may not sum as expected due to rounding. 

4 Glosser et al. (2018) did not require corrections for clustering or multiple comparisons nor difference-in-differences adjustments. Findings from the 18-month follow-up survey, 
with imputed cases removed, were presented by the author in response to a WWC author query. The p-value for receipt of a credential from any source was calculated by the 
WWC since the author provided the results of a one-tailed test. This study is characterized as having a statistically significant positive effect on industry-recognized credential, 
certificate, or license completion because the estimated effect is positive and statistically significant. The study is characterized as having an indeterminate effect on short-term 
earnings because the estimated effect reported is not statistically significant. 


> Modicamore et al. (2017) did not require corrections for clustering nor difference-in-differences adjustments. A correction for multiple comparisons was needed but did not 
affect whether any of the contrasts were found to be statistically significant. The p-value for employment in the first year after program completion was not presented in the 
original study and was calculated by the WWC. Authors did not report standard deviations of earnings outcomes; however, they did report the effect size using the Hedges’ 

g formula and conducted a Benjamini-Hochberg correction for multiple comparisons. This study is characterized as having a statistically significant positive effect on (a) 
industry-recognized credential, certificate, or license completion, (b) short-term employment, and (c) short-term earnings because the estimated effect is positive and statistically 
significant. 

° Anderson et al. (2017) did not require corrections for clustering or multiple comparisons nor difference-in-differences adjustments. The outcomes for earning any credential 
from a college and earning more than 12 credits were aggregated across subsamples by the WWC, and the p-values for these outcomes were calculated by the WWC based 
on the aggregated sample. This study is characterized as having a statistically significant positive effect on industry-recognized credential, certificate, or license completion 
because the estimated effect is positive and statistically significant. This study is characterized as having an indeterminate effects on credit accumulation because the estimated 
effect is not statistically significant. 


For more information, please refer to the WWC Procedures Handbook, version 4.0, page 22. 


In What Context Was I-BEST Studied? 


The following section provides information on the setting of 
the three studies of I-BEST that meet WWC standards, and a 
description of the participants in the research. This 
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information can help educators understand the context in 
which the studies of -BEST were conducted, and determine 
whether the program might be suitable for their setting. 


3 studies, 45,413 students in Connecticut, Georgia, Illinois, 
Kansas, Kentucky, Louisiana, Maryland, Texas, and Washington 


Districts: Urban, suburban, and rural settings 
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Details of Each Study that Meets WWC Standards 


This section presents details for the studies of I-BEST that 
meet WWC standards. These details include the full study 
reference, findings description, findings summary, and 
description of study characteristics. A summary of domain 
findings for each study is presented below, followed by a 
description of the study characteristics. These study-level 
details include contextual information about the study 
setting, methods, sample, intervention group, comparison 
group, outcomes, and implementation details. For addi- 
tional information, readers should refer to the original 
studies. 


Research details for Glosser et al. (2018) 


Glosser, A., Martinson, K., Cho, S. W., & Gardiner, K. (2018). 


Washington State’s Integrated Basic Education and Skills 


Training (I-BEST) Program in three colleges: Implementa- 
tion and early impact report (OPRE Report No. 2018-87). 
Washington, DC: Office of Planning, Research, and Eval- 
uation, Administration for Children and Families, U.S. 
Department of Health and Human Services. 


Findings from Glosser et al. (2018) show evidence of a 
statistically significant positive effect of I-BEST industry-rec- 
ognized credential, certificate, or license completion. The 
study showed evidence of an indeterminate effect on short- 
term earnings (Table 4). These findings are based on an 
outcome analysis that includes 455 students. 


Table 4. Summary of findings from Glosser et al. (2018) 


Meets WWC group design standards without reservations 


Study findings 
. Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
Industry-recognized credential, certificate, or 424 students 0.49 +19 Yes 
license completion 
Short-term earnings 455 students -0.03 -1 No 


Table 5. Description of study characteristics for Glosser et al. (2018) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Comparison 
group 


Outcomes and 
measurement 


Meets WWC Group Design Standards Without Reservations. This is a randomized controlled trial (RCT) with low 
attrition.” For more information on how the WWC assigns study ratings, please see the WWC Procedures and Standards 
Handbooks (version 4.0) and WWC Standards Briefs, available on the WWC website. 


The study took place at three public community colleges in Washington state: Bellingham Technical College, Everett 
Community College, and Whatcom Community College. 


/-BEST program applicants first took the Comprehensive Adult Student Assessment System (CASAS) math and reading 
assessment. Students with CASAS scores within the eligibility range of their desired program then met with /-BES7 staff to 
confirm their interest and address program-specific requirements, such as tuberculosis screening for applicants to nursing 
assistant programs. Students who consented to be in the study were then given a baseline questionnaire, completed a 
baseline information form, and were randomly assigned by an online system to either receive /-BEST or to serve ina 
comparison group. Altogether, 632 study participants were randomly assigned to intervention or comparison conditions 
between November 2011 and September 2014. The baseline sample included 315 students in the intervention group and 
317 students in the comparison group. One student in the comparison group dropped out of the study. The study included 
data that were imputed using a method that did not meet WWC standards. In response to a WWC author query, the author 
provided findings from the study’s 18-month follow-up survey with imputed data removed. The analytic sample for the 
credential receipt outcome includes 214 students in the intervention group and 210 students in the comparison group. The 
analytic sample for the working in a job paying $12 an hour or more outcome includes 235 students in the intervention 
group and 220 students in the comparison group. 


Thirty-one percent of students had yet to obtain a high school diploma or equivalent, and 58 percent of students were 
female. Approximately 10 percent reported having attended one or more years of college. Study participants also had lower 
incomes and were older than traditional college students: almost two-thirds of students (63 percent) were age 25 or older. 
Slightly more than half (55 percent) were non-Hispanic White, and about one quarter (26 percent) identified as Latino or 
Hispanic. Two-thirds (67 percent) of study participants were not working at the time of random assignment, with only 13 
percent working 35 hours or more per week. 


The /-BEST program includes courses that are part of a structured pathway. Integrated team-teaching of basic skills and 
occupational skills was done in most courses on the pathway. Team teaching took several forms. In some courses, the basic 
skills instructor sat in class with students and stopped the occupational instructor to ask clarifying questions or to explain 

a concept further. In other courses, the basic skills instructor would either deliver a designated portion of the instruction or 
would jointly deliver instruction with the occupational instructor. 


I-BEST students had access to dedicated advisors, called navigators, who provided guidance on academic issues, helped 
students navigate the college’s procedures, and helped with career planning. |-BEST also provided “fill the gap” funds for 
books, tools, other course materials, and transportation. This funding ensured that all members of the intervention group 
would pay no tuition. 


Comparison group members could not access /-BEST programs and courses at the three colleges; however, they could 
access other education and training opportunities available to them, including non-/-BEST courses and /-BEST programs 
at other colleges. Both intervention and comparison group members could access general college advising, tutoring, and 
financial aid services that were available to all students at the colleges. Both intervention and comparison group members 
could potentially access financial support through Pell grants, Washington State’s Opportunity Grants, Washington's Basic 
Food Employment and Training (BFET) program, veteran’s benefits, and Temporary Assistance for Needy Families (TANF), 
depending on eligibility. Both intervention and comparison group members could access their college’s employment and job 
placement services designed to help program completers find jobs. Whether they enrolled in college classes or not, they 
also could access other employment assistance in the community, such as the job search and job readiness services at 
local American Job Centers. 


Study authors reported findings on four outcome measures that are eligible for review under the Postsecondary Career 
and Technical Education topic area. These findings include receipt of a credential from any source (industry-recognized 
credential, certificate, or license completion domain), working in a job paying $12 an hour or more (short-term earnings 
domain); academic and workforce credits earned (credit accumulation domain); and received an associate's degree or 

higher (postsecondary degree attainment domain). 


Because the author imputed data using an unacceptable method for the academic and workforce credits earned outcome, 
as well as the received an associate's degree or higher outcome, these outcomes were not able to meet WWC standards. 
The author shared findings from the 18-month follow-up survey after removing imputed data points from the calculations. 
Survey outcomes that meet WWC standards included receipt of a credential from any source and working in a job paying 
$12 an hour or more. 


The study also reported findings for receipt of a credential broken down by whether the credential was received from a 
college, another education-training institution, or from a licensing or certification body. Summaries of these findings are 
available on the WWC website (https://whatworks.ed.gov). The supplemental findings do not factor into the intervention’s 
rating of effectiveness. 


Outcomes and 
measurement 
(continued) 


Additional 
implementation 
details 


Other findings were included in the study that were not eligible in the Postsecondary Career and Technical Education topic 
area, including enrollment in college, type of organization providing a credential, enrollment in developmental courses, 
completion of developmental courses, receipt of a certificate or degree, employment in a job requiring at least mid-level 
skills or higher, and a number of attitudinal and perception measures. 


I-BEST was designed by the Washington State Board for Community and Technical Colleges (SBCTC) and operates at all 
34 public community and technical colleges in the state. The three colleges included in this study received additional funding 
for program enhancements from the Open Society Foundations. 


Research details for Modicamore et al. (2017) 


Modicamore, D., Lamb, Y., Taylor, J., Takyi-Laryea, A., Findings from Modicamore et al. (2017) show evidence of a 
Karageorge, K., & Ferroggiaro, E. (2018). Accelerating statistically significant positive effect of I-BEST, called 
Connections to Employment: Final evaluation report. Accelerating Connections to Employment in this study, on 
Fairfax, VA: ICF International. industry-recognized credential, certificate, or license 


completion; short-term employment; and short-term 
earnings (Table 6). These findings are based on an outcome 
analysis that includes 2,064 students. 


Table 6. Summary of findings from Modicamore et al. (2017) 


Meets WWC group design standards without reservations 


Study findings 
Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
Industry-recognized credential, certificate, or 1,049 students 0.45 +17 Yes 
license completion 
Short-term employment 2,064 students 0.27 +10 Yes 
Short-term earnings 2,064 students 0.02 +{ Yes 


Table 7. Description of study characteristics for Modicamore et al. (2017) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Comparison 
group 


Meets WWC Group Design Standards Without Reservations. This is a randomized controlled trial (RCT) with low 
attrition. 


The study took place at nine sites, including six sites in Maryland (Anne Arundel County, Baltimore City, Baltimore County, 
Montgomery County, Prince George’s County, and the Upper Shore), one site in Connecticut (New Haven), one site in 
Georgia (Atlanta), and one site in Texas (Austin). 


Study participants were recruited by a community college, One-Stop center, or both. In order to be eligible for the study, 
incoming students had to (a) possess sufficient basic skills to benefit from the Accelerating Connections to Employment 
(ACE) training, (b) have sufficient English language proficiency to participate in ACE training, and (c) have no other 
impediments to the successful receipt of ACE training. Eligible students met with a career counselor who administered a 
skills assessment (CASAS or Test of Adult Basic Education-TABE), assessed career interests, and identified needs for 
training. If a student met the established range of CASAS or TABE scores to participate in a given program, they were 
invited to participate in the study. Students who consented to be in the study were then randomly assigned to either 
participate in ACE training (n=1,175) or seek alternative services on their own at the One-Stop center (n=993). Prior to 
randomization 83 veterans were automatically enrolled in ACE training, as required by the Jobs for Veterans Act of 2002. 
These veterans were not enrolled in the study. The sample loss after random assignment (or attrition) was within the 
acceptable threshold for the review. 


ACE study participants were recruited across five cohorts and randomized prior to the start of the intervention. In early 
cohorts, sites were allowed to randomize more study participants into the intervention group than the comparison group to 
promote enrollment. Later cohorts entailed balanced random assignment. Adjusted weights were therefore included in the 
analyses for each cohort to address differences in balance between study conditions. 


Random assignment resulted in an intervention group of 1,175 students and a control group of 993 students. Some 
outcomes were drawn from Unemployment Insurance (Ul) records in each state, so attrition was minimal on employment 
and earnings outcomes; however, UI data were not available for the final quarter of the ACE program in quarter 4 of 2015. 
Study authors administered a survey to study participants one year and two years after program completion. The year 

1 follow-up survey had 691 survey respondents in the intervention condition and 521 in the comparison condition. The 
year 2 follow-up survey had 280 survey respondents in the intervention condition and 207 respondents in the comparison 
condition. 


The 2,168 study participants were 71.3 percent African American, 15.4 percent White, 1.2 percent Native American, 

2.5 percent Asian, 0.5 percent Native Hawaiian, and 9.1 percent other. Moreover, 11 percent of study participants were 
Hispanic, and 70.5 percent of study participants were female. Almost two-thirds of study participants (64.9 percent) were 
unemployed at baseline, 34.5 percent were employed, and 0.6 percent were not in the labor force. On average, study 


participants were 35.5 years of age. Almost half of study participants (48.2 percent) had a high school diploma or GED, 18.0 


percent had yet to obtain a high school diploma, 28.8 percent had some college or an Associate’s degree, and 5.0 percent 
had a Bachelor's degree or higher. 


ACE is based in part on Washington State’s -BEST model. Like |-BEST, ACE provides integrated basic skills and 
occupational skills training, with at least 50 percent of total training hours using a co-teaching model. ACE also engages 
employers, industry partners, and Workforce Investment Boards (WIBs) to assess labor market demands, determine basic 
skills requirements, and design programs that prepare job seekers for high-demand occupations. ACE also provides a 
career navigator to students. Initially, the ACE career navigator was involved in all aspects of recruitment, onboarding, job 
readiness training, and job placement. Later, sites added a job developer position, which afforded career navigators more 
time to focus on forming strong relationships with participants and guiding them through the training program. 


Comparison group members had access to alternative services at the WIB. As with any non-ACE WIB customer, control 
group members generally had to navigate these alternative services on their own. 
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Research details for Anderson et al. (2017) 


Anderson, T., Kuehn, D., Eyster, L., Barnow, B., & Lerman, 
R. I. (2017). New evidence on integrated career pathways. 
Washington, DC: Urban Institute. Retrieved from 
https://www.urban.org/sites/default/files/ 
publication/91436/ao final impacts.pdf 


Findings from Anderson et al. (2017) show evidence of a 
statistically significant positive effect of I-BEST, implemented 
in this study as Accelerating Opportunity, on industry- 


recognized credential, certificate, or license completion, 
and a statistically significant negative effect on credit 
accumulation (Table 8). These findings are based on an 
outcome analysis that includes 42,894 students. The findings 
and research details summarized for this study come from 
six related citations, including the primary study listed 
above. See the References section, which begins on page 13, 
for a list of all related publications. 


Table 8. Summary of findings from Anderson et al. (2017) 


Meets WWC group design standards with reservations 


Study findings 
Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
Industry-recognized credential, certificate, or 42,894 students 0.49 +19 Yes 
license completion 
Credit accumulation 42,894 students -0.03 -1 No 
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Table 9. Description of study characteristics for Anderson et al. (2017) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Comparison 
group 


Outcomes and 
measurement 


Additional 
implementation 
details 


Meets WWC Group Design Standards With Reservations. This is a quasi-experimental design (QED) with baseline 
equivalence established on the analytic sample.° 


The intervention was delivered in community college and adult education settings in four states: Illinois, Kansas, Kentucky, 
and Louisiana. 


The study used a quasi-experimental design. Authors used administrative data to match 4,760 students enrolled in a 
for-credit course at a participating Accelerating Opportunity college to other students in the state who were drawn from the 
same recruitment pool. In Illinois and Louisiana, colleges recruited mainly from the adult education population. In Kansas, 
recruitment focused on students in adult education and current career and technical education (CTE) students; in Kentucky, 
recruitment focused on students in adult education and developmental education students. Propensity score matching 
procedures were used to match students enrolled between calendar years 2012 to 2014 in Illinois, Kansas, and Kentucky. 
In Louisiana, the analytic sample includes students enrolled between fall 2012 and summer 2015. Seventeen variables 
were used in the propensity score matching model, including student demographics, prior educational experience, prior 
employment status, socioeconomic status, local job conditions, and academic performance as measured by basic skills and 
other postsecondary assessments. Authors used a propensity score matching procedure that matched comparison group 
students within a range of propensity scores to an intervention group member, so each Accelerating Opportunity student 
may have multiple comparison students. 


The sample sizes for the intervention and comparison groups, respectively, are: 867 and 4,129 in Illinois, 1,698 and 12,595 
in Kansas, 1,356 and 18,794 in Kentucky, and 440 and 3,015 in Louisiana. The total sample size is 42,894, with 4,361 
students in the intervention group and 38,533 students in the comparison group. 


The analytic sample is 56 percent female, 62 percent White, 19 percent Black, and 11 percent Hispanic. Thirty-six percent 
of the analytic sample was eligible to receive a Pell grant. 


Accelerating Opportunity is based on the /-BEST model, and is designed to help low-skilled students earn occupational 
credentials, obtain employment, and sustain careers. Community and technical colleges that were in the intervention 
condition developed or modified existing programs that offered career pathways for in-demand jobs. A major component of 
Accelerating Opportunity was integrated instruction, where both basic skills and CTE instructors taught the same class with 
at least 25 percent overlap. Students also received additional services, including tutoring, academic advising, college 
navigation, job search assistance, job placement, and case management. Accelerating Opportunity programs partnered with 
Workforce Investment Boards (WIBs) to connect students to employment. 


The comparison condition entailed standard, business-as-usual instruction and support. The comparison group students 
were drawn from the same recruitment sources—including adult education, developmental education, or CTE—as the 
intervention group, but they did not have the opportunity to participate in Accelerating Opportunity. 


Study authors reported findings on two outcome measures that are eligible for review under the Postsecondary Career 
and Technical Education topic area. These outcomes include earned any credential from a college (industry-recognized 
credential, certificate, or license completion domain) and earned more than 12 credits (credit accumulation domain). 


The study also reported supplemental findings for Illinois, Kansas, and Kentucky programs. Supplemental findings in 

the industry-recognized credential, certificate, or license completion domain include the average number of credentials 
received and percentage of students receiving credentials. Supplemental findings in the credit accumulation domain include 
total credits earned and the percentage of students who received more than 12 credits. Summaries of these findings are 
available on the WWC website (https://whatworks.ed.gov). The supplemental findings do not factor into the intervention’s 
rating of effectiveness. 


Employment and earnings outcomes for Louisiana and Illinois for three and four quarters, respectively, after program entry 
were eligible to receive a rating of Meets WWC Group Design Standards with Reservations; however, these outcomes were 
measured less than one year after the earliest program completion and were therefore ineligible for review as short-term 
outcomes according to the Postsecondary Career and Technical Education protocol. Baseline equivalence could not be 
established on subsequent data collection time points. Labor market outcomes for Kansas and Kentucky do not meet 
WWC group design standards because baseline equivalence could not be established on the sample used in the analysis. 
In Louisiana, the intervention and comparison group students were not balanced on Pell eligibility, so education outcomes 
from this state do not meet standards. 


No additional implementation details were reported. 
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4 The descriptive information for this intervention comes from 
SBCTC’s I-BEST website, as well as Glosser et al. (2018), Modicamore 
et al. (2017), and Anderson et al. (2017). The What Works 
Clearinghouse (WWC) requests developers review the intervention 
description sections for accuracy from their perspective. The WWC 
provided the developer with the intervention description in October 
2019 and the WWC incorporated feedback from the developer. 
Further verification of the accuracy of the descriptive information 
for this intervention is beyond the scope of this review. 


5 Absence of conflict of interest: This intervention report includes 
a study conducted by staff from Abt Associates (Glosser et al., 
2018). Because Abt Associates is a contractor that administers the 
WWC, theis study was reviewed by staff members from a different 
organization. 

6 The literature search reflects documents publicly available by 
August 2019. Reviews of the studies in this report used the 
standards from the WWC Procedures and Standards Handbook 
(version 4.0) and the Postsecondary Career and Technical 
Education review protocol (version 4.0). The evidence 
presented in this report is based on available research. Findings 
and conclusions could change as new research becomes 
available. 

7 Attrition rates were negligible for Glosser et al. (2018): only one 
student dropped out of the control group. 

8 Attrition rates were low according to WWC standards. Attrition 
rates for the first-year follow-up survey were low, and were 
deduced from baseline equivalence tables provided in the report. 
Findings from the second-year follow-up survey had high attrition 
and did not demonstrate baseline equivalence; therefore, they did 
not meet WWC group design standards. 

° WWC reviewers were able to establish baseline equivalence on 
at least one outcome for all four states. These findings receive a 
WWC rating of Meets Standards With Reservations. In Louisiana, 
the intervention and comparison group students were not 
balanced on Pell eligibility, so education outcomes from this state 
do not meet standards. Louisiana is included in the aggregate 
education outcomes since the aggregated sample is balanced on 
required baseline measures. The analytic sample sizes for the 
labor market outcomes in Kansas and Kentucky do not match up 
with the samples for which the WWC had baseline information. 
Therefore, in Kansas and Kentucky, only the education outcomes 
meet standards and are coded in this review. The follow-up period 
for the percent employed outcome for Illinois and Louisiana 
differs, so the results are not aggregated. In response to a query, 
the author reported that standard deviations of continuous 
outcome variables are not available. 
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